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Abstract 


Innovation in instructional technology has contributed to the rapid 
implementation of technology-driven instructional platforms, particularly in 
developmental math coursework. Prior research has shown that instructional environment 
and classroom experience influence student development and outcomes. Consequently, 
when courses transition to technology-driven instruction, a logical concern on the part of 
faculty and administrators is the effect on the quality of the academic experience among 
students. Under a hybrid emporium model, students primarily receive instruction from a 
computer-based platform rather than from a faculty member delivering content in front 
of the classroom. This paper examines how students experience a newly adopted, hybrid 
emporium model for developmental math coursework. We conducted focus groups with 
students at six public colleges in Tennessee and find that students enrolled in hybrid 
emporium developmental math courses reported that the instructional model contributes 
to lowered barriers to math by increasing cognitive and social accessibility. In spite of 
prior academic challenges, students perceived math content and their faculty to be more 
accessible in the computer-driven model than in traditional lecture classes. We discuss 
these findings in light of recent research suggesting technology-driven instruction does 
not improve math performance. 
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1. Introduction 


Roughly half of all first-year college students arrive on campus underprepared for 
postsecondary academic work (Chen, 2016; Center for Community College Student 
Engagement, 2016). Most institutions offer additional coursework for these students, 
referred to as developmental, remedial, college preparation, learning support, or skills 
courses. These developmental courses are often the first academic environments that 
many students encounter in college. Unfortunately, enrollment in traditional 
developmental courses does not guarantee academic success. Students who enroll in 
remedial courses to improve their math, reading, or writing skills remain at risk for 
dropout and disengagement from college. Despite evidence of some positive academic 
outcomes for students who enroll in such courses (Bahr, 2008; Boatman & Long, 2018), 
other evidence suggests that remediation presents obstacles to the pursuit of credit- 
bearing coursework or contributes to the phenomenon of community college “cool out” 
in which students lower their academic aspirations (Bailey et al., 2016; Bailey, Jeong, & 
Cho, 2010; Deil-Amen & Rosenbaum, 2002; Edgecombe, 2011; Jaggars & Bickerstaff, 
2018). 


In response to students struggling through traditional developmental courses, 
institutions and, in some cases, state systems have crafted new approaches to remediation, 
many of which harness technology to guide students through competency-based learning 
modules (Arum & Roksa, 2011; Fay, 2017; Herreid & Schiller, 2013; Lage, Platt, & 
Treglia, 2000). The use of technology in online and in-class postsecondary instruction 
has grown rapidly over time, particularly in the remediation of mathematics skills (Allen 
& Seaman, 2007, 2010; Means, Bakia, & Murphy, 2014). Technology can be used to 
expand, strengthen, and create efficiencies in the delivery of developmental math courses 
(Epper & Baker, 2009). For example, technology-focused reforms can either accelerate 
students through their developmental course sequence faster by compressing the material 
into fewer semesters or they can allow students to move through the developmental 
material at their own pace, both of which have the potential to positively impact students’ 
experiences and academic outcomes (Fong & Visher, 2013). 


One popular strategy for incorporating technology into remedial mathematics 
courses is known as the hybrid emporium model, which eliminates lecture, uses 
interactive software, and provides personalized, on-demand assistance for students. At 
the beginning of the course, students complete a diagnostic assessment that identifies their 
math strengths and weaknesses. Students then spend much of their class time in a 
computer lab learning the course content online at their own pace, with faculty serving 
more as tutors who deliver individualized instruction rather than as lecturers. Faculty 


track student progress online and answer questions individually, both online and in the 
lab. The course content is divided into small blocks, or modules, and is taught using 
tutorials, practice exercises, and quizzes and tests. When students fail to master a concept, 
they are allowed to complete the module again, with new problems replacing the old. 
When students complete the number of modules assigned during the diagnostic 
assessment, they have passed the developmental math course and may typically move on 
to their gateway college-level math course. 


Instructional redesigns generally may yield both benefits and challenges for 
students. Technology-driven redesigns may offer individualization of instruction but may 
lower academic engagement with faculty due to the nature and quality of faculty-student 
engagement and content delivery. This study seeks to illuminate students’ perceptions of 
the benefits and challenges of learning developmental math under a technology-driven 
instructional approach. Using qualitative data from student focus groups at six public 
colleges in Tennessee, we analyze students’ reactions to the hybrid emporium model. We 
address the primary research question: What do students report as the impacts of 
developmental math instruction through the hybrid emporium model on their academic 
and psychosocial experiences in college courses? While other research (see Boatman, 
2019) estimates the academic effects of using the hybrid emporium model, in this paper 
we explore the student experience in a technology-driven classroom and consider how it 
shapes students’ academic experiences more broadly. With the rapid adoption of the 
hybrid emporium model across college campuses, student voices, particularly those of 
individuals in need of developmental coursework, are underrepresented in the literature 
examining instructional changes. Our use of student focus groups brings to light the 
educational experiences of academically at-risk students who are taught using this 
increasingly popular instructional model. 


We find that students enrolled in hybrid emporium developmental math courses 
reported that their classroom experience breaks down barriers to math learning through 
greater cognitive and social accessibility. In spite of prior academic challenges, these 
students perceived math content and their faculty to be more accessible in a hybrid 
emporium course than in traditional lecture courses. Their comments focused on access 
to academic content and a more supportive faculty-student relationship. However, despite 
the cognitive and social benefits students perceived after participating in a hybrid 
emporium course, recent research suggests that the academic outcomes for students 
enrolled in these courses are not necessarily improved (Boatman, 2019; Kozakowski, 
2019). We conclude by discussing this tension and its potential impact on students’ 
subsequent academic success. 


2. Background: Technology-Driven Instruction in 
Developmental Math 


Math curricula in the United States have increased in their cognitive demand over 
the past 100 years (Blair, Gamson, Thorne, & Baker, 2005). Whereas basic arithmetic 
comprised the bulk of the mathematics curriculum prior to the 20" century, demands for 
improved mathematics and science education at the beginning of the Cold War and into 
the 1990s have yielded more complex mathematics standards and greater requirements in 
compulsory and postsecondary education. This increase in mandatory math course-taking 
has not been mirrored in the evolution of other academic subjects. 


It is not surprising, therefore, that 37 percent of U.S. teenagers report that math is 
their most difficult subject (Saad, 2005) and that students are more likely to fail 
developmental math than any other course in higher education (Le, Rogers, & Santos, 
2011). Multiple factors contribute to math being a demanding subject for students 
throughout the educational pipeline. Learning math is cognitively demanding in that it 
necessarily employs various learning techniques, leverages visual and spatial ability, and 
uses specific cognitive strategies and critical thinking skills (Higbee & Thomas, 1999). 
This cognitive demand contributes to mental burden, which often manifests as math 
anxiety and may threaten academic self-concept, attitudes toward math, confidence in 
one’s ability to learn math, perceptions of math’s usefulness, and locus of control (Higbee 
& Thomas, 1999). Math anxiety constitutes a major barrier to success in math courses 
(Ramirez, Shaw, & Maloney, 2018). 


In an effort to address these barriers, the U.S. Department of Education and What 
Works Clearinghouse issued a report (Bailey et al., 2016) outlining six recommendations 
for practitioners in assessing, advising, encouraging, and educating students struggling 
with math and in need of college remediation. One recommendation encourages 
differentiation of the curriculum such that students arriving with varied levels of 
preparation receive individualized instruction alongside targeted supports. Technology is 
a primary tool that can be harnessed to personalize student learning in this way. 
Technology-driven remediation presents an opportunity to individualize and accelerate 
learning, as well as to increase academic intensity for struggling students (Epper & Baker, 
2009). 


Individualizing and Accelerating Learning 


The modularized format of teaching in a hybrid emporium classroom allows 
students to review material and demonstrate mastery before moving on to the next topic 


(Bickerstaff, Fay, & Trimble, 2016). The adaptive nature of the software targets students’ 
areas of need, allowing for increased time spent practicing in these areas and less time 
spent on topics already mastered. Consequently, students have the flexibility to take more 
time on topics when needed. Additionally, students, particularly the most vulnerable, may 
experience increased individualized academic support from instructors, as instructors 
have more time to respond to students’ immediate questions and needs (Twigg, 2011). 
Individualizing instruction may also lead to more autonomy and self-efficacy in the 
classroom, which can, in turn, promote students’ intrinsic motivation to perform tasks, 
sense of agency, and belief in their own competence (Ryan & Deci, 2000; Garcia & 
Pintrich, 1996). Evidence suggests that faculty encouragement of student autonomous 
action predicts greater self-efficacy and motivation in related tasks and learning (Black 
& Deci, 2000; Overall, Deane, & Peterson, 2011). Individualized learning also stands to 
save students time and money, as they need only retake the modules they have not yet 
passed rather than retake an entire course (Epper & Baker, 2009). This allows students to 
move through their courses more quickly. 


Increasing Academic Intensity 


In addition to offering students individualized instruction and the freedom to 
move through their developmental math courses more quickly, technology-driven courses 
like those using the hybrid emporium model are also potentially able to cover more 
content than a uniform semester-long course. Instructors may more efficiently help their 
students master skills and concepts through individualized learning programs that 
monitor student progress and adjust the content delivered accordingly (Hagerty & Smith, 
2005). Students who are able to master basic course concepts are able to move on to more 
advanced material without waiting on their peers. These environments can encourage 
engagement in higher-order thinking and learning, as the technology is able to push 
students in the areas where they are least proficient (Babcock & Marks, 2010). This 
differentiation creates a classroom environment in which instructors may be assisting 
with a wide variety of mathematics topics, from the very basic to the advanced. In this 
sense, technology-driven instruction presents an opportunity for both increased rigor and 
pedagogical change. 


Prior Research on Academic Outcomes 


The academic benefits of accelerating learning and increasing academic intensity 
may explain some of the positive impacts of technology-driven instruction reported in the 
literature. Hagerty and Smith (2005) evaluated the efficacy of a particular web-based 
software in mathematics instruction by comparing the outcomes in four technology- 


driven college algebra classrooms with those in four traditionally taught classrooms by 
the same instructors in the same semester. They found evidence of short- and long-term 
gains: Students in the technology-driven classrooms had more skills growth during the 
semester and performed better on a standardized exam in their junior year (Hagerty & 
Smith, 2005). In a study of technology-driven instruction at 32 institutions that redesigned 
86 developmental math courses, Twigg (2013) concluded that 83 percent of the 
redesigned courses showed significant improvements in end-of-course exams and 
subsequent course grades over traditional developmental math courses. And in a 
randomized controlled trial, Taylor (2008) found that technology-facilitated instruction 
reduced math anxiety and improved attitudes toward math among students after a 
semester, compared to control group students who took a traditional lecture-based course 
(Taylor, 2008). 


However, not all of the research on technology-driven instruction has found 
positive impacts on student outcomes and attitudes. Common critiques of the approach 
include lower comparative responsiveness, less accountability, and the inherent 
distraction of self-driven, computer-based work. Students surveyed about computer- 
driven curricular redesign have reported a lack of instructor support (Epper & Baker, 
2009). In the programs they profiled, students reported that they did not always want to 
teach themselves, and they found it difficult to collaborate with peers, as they were often 
working through different modules. While students may be open to using technology for 
formulaic tasks such as email or submitting assignments through online course 
management systems, research suggests that students may prefer person-to-person 
contact for more complex tasks such as course instruction and academic advising 
(Kalamkarian & Karp, 2017). 


Another criticism of technology-driven instruction is that instructional modules 
are not consistent in the number of assignments given, amount of information covered, 
or difficulty of content. Research has shown that some modules are more content-heavy 
or more difficult for students than other modules and thus have lower pass rates (Fay, 
2017). An evaluation of computer-mediated developmental math in the Virginia 
Community College System (VCCS) found that more than 80 percent of students had not 
completed the four required modules during a semester, and 42 percent had not completed 
any modules at all (Bickerstaff et al., 2016). Faculty have also reported a lack of deep 
learning on the part of students, perhaps because of the repetitive nature of practice and 
the style of assessment. They worry students may be able to perform well on the tests but 
cannot translate these important skills to other contexts, including higher-level math 
courses. Faculty also express concerns that institutions rely too much on technology to 
teach and support their neediest students, who might in particular benefit from more 
person-to-person interaction (Bonham & Boylan, 2011). 


These issues may contribute to the negative findings on academic performance of 
students in hybrid developmental math courses compared to those in traditional courses. 
Research on students in hybrid emporium-style courses from community colleges in 
Tennessee (Boatman, 2019) and Kentucky (Kozakowski, 2019) has shown, in 
comparison to traditional courses, lower pass rates among students for subsequent college 
math courses, fewer number of credits completed over time, and lower associate degree 
completion rates. While these effects differ by student subgroup, the results overall 
suggest that the hybrid emporium model is not leading to improved academic 
performance for students in gateway college-level math courses. 


The ultimate goal of instructional innovation is to improve both the academic 
development and classroom experience of students. While there is existing research on 
the impact of the hybrid emporium model on subsequent academic performance, there is 
little research examining how students experience technology-driven instruction. Despite 
mixed evidence of the model’s effectiveness and a lack of data on student experience, the 
hybrid emporium model continues to be rapidly adopted across the United States for the 
facilitation of developmental mathematics learning. It is therefore key to understand how 
students experience this technology-driven instructional approach in order to inform 
implementation. Our research examines how students report experiencing a technology- 
driven instructional model in developmental math, with the goal of providing insight into 
why programs that use the model may or may not achieve their intended outcomes and 
how they might be improved to better serve future cohorts of students. 


3. Conceptual Framework 


Technology-driven instruction may change the way students experience 
developmental math courses by altering their learning environment. This change may 
influence the cognitive and social accessibility of math. Learning math in an instructional 
setting is both cognitive, in the sense of acquiring specific math skills, and social, as 
relationships that students form with actors in the classroom and the emotions they 
experience there may affect how students learn and their purpose and orientation. 
Students who perceive their faculty members as approachable and available have greater 
confidence in their academic skills and greater motivation (Komarraju, Musulkin, & 
Bhattacharya, 2010). Thus, while instructional change is implemented largely to increase 
students’ cognitive accessibility to the material, technology-driven instruction’s influence 
on the nature of teaching and learning in college-level developmental math could also 
lead to changes in students’ attitudes and perceptions toward math and academic 
engagement. Taken together, the ability to access material continuously along with one- 
on-one attention from instructors and changes in the faculty-student power dynamic may 
fundamentally alter the relationships that students form with actors in their classrooms, 
which may ultimately serve to reshape their beliefs and feelings about math. 


Cognitive Accessibility 


The differences between the hybrid emporium model and traditional math 
instruction, especially in terms of instructional delivery and student access to course 
materials, may change the cognitive accessibility of math. Under the hybrid emporium 
model, students have access to math instruction at any time. Through videos, quizzes, and 
practice problems, the modules provide abundant opportunities for practice. 
Instantaneous grading and error identification provide immediate feedback. Whereas in 
other classrooms, questions must be held for class time, office hours, or tutoring, students 
in a hybrid emporium course can receive help from the online platform in real time or 
from the instructor via email or mobile application before the next class. Additionally, 
the online platform may present alternative strategies for solving problems and learning 
material, which provides students with a valuable complementary perspective that they 
might not receive in a traditional classroom. 


Social Accessibility 


Technology-facilitated instruction may also change the social accessibility of 
math, or the relationships that students form with actors in the math classroom, ultimately 


shaping their affective experience of learning math. Whereas in a traditional classroom, 
teachers spend the bulk of their time addressing the entire class, in a hybrid emporium 
classroom, teachers float between students, spending time with individual students to 
address their personal questions or to check on their progress, thereby altering the student- 
faculty relationship. This model fundamentally changes the power dynamics of a 
traditional classroom. Whereas in a lecture-driven classroom, instructors often function 
as the gatekeepers and purveyors of knowledge, in a hybrid emporium classroom, the 
online platform delivers material to students. This diffusion of content authority and the 
practice-driven nature of the material may ultimately have a democratizing effect. 


Longstanding evidence suggests that faculty-student interpersonal relationships 
shape students’ college experiences by contributing to skill development and engagement 
(Astin, 1977; Pascarella & Terenzini, 1976; Umbach & Wawrzynski, 2005). The 
reoriented classroom under the hybrid emporium model may offer faculty the opportunity 
to engage with students in more meaningful ways. According to prior research, students 
report higher levels of engagement and learning when faculty interact with students and 
use active and collaborative learning techniques (Umbach & Wawrzynski, 2005). 
Through individualized engagement with faculty, peers, and tutors, students may develop 
more autonomy (Chickering & Reisser, 1993). Changing the nature and frequency of 
interactions with faculty may, in turn, change students’ perceptions, engagement, and 
development in the classroom. 


Alternatively, there are some circumstances in a hybrid emporium classroom in 
which students may have fewer opportunities to engage with faculty or may do so in less 
meaningful ways. The expectation in a hybrid emporium classroom is that faculty and 
tutors circulate to answer student questions. Under this model, students may perceive that 
they have less contact time or a more tenuous connection to their instructor than under 
the traditional lecture model. Even in the context of their one-on-one interactions, 
students and faculty may find it more difficult to establish rapport and common ground 
in the absence of direct instruction. Student-faculty engagement may be particularly 
important for students with the lowest standardized test scores (Carini, Kuh, & Klein, 
2006). Given the correlation between entrance exam scores and math placement, better 
engagement of developmental students could be particularly helpful for students’ long- 
term outcomes. Community college students generally have lower levels of engagement 
with faculty, and their interactions with faculty are concentrated during class time 
(Chang, 2005). Chang recommends that instructors adapt their instruction to more 
frequently engage in discussion with students around material. Such an instructional 
adjustment may be accomplished through the individualized interactions built into a 
hybrid emporium classroom. 


Students may also perceive differences in their engagement with their peers in the 
technology-based classroom. While working side by side at computers, students may 
communicate and help one another in different ways than they would while sitting in a 
lecture. They may make more substantive connections with classmates as they navigate 
the computer platform and negotiate with difficult material. Alternatively, while facing 
screens and working independently with expectations of silence, some students may find 
fewer or poorer opportunities to connect with peers. 


4. Study Design and Sample 


Our study uses a case study approach to answer the research question: What do 
students report as the impacts of developmental math instruction through the hybrid 
emporium model on their academic and psychosocial experiences in college courses? We 
collected our data in two phases: survey data collection and campus site visits. In the first 
phase, we administered a 35-question online survey for administrators and instructors 
during the summer of 2016 using the Qualtrics online platform. Survey questions focused 
on topics related to initial implementation and ongoing use of technology-driven 
instruction, including the degree to which instructors leverage the instructional platform, 
the logistics of implementing the model, and the proportion of incoming students who are 
required to take developmental math. The survey was distributed via email to 
administrators and instructors who serve as Tennessee Board of Regents developmental 
math coordinators at each institution, in order to seek perspective from the individuals 
most intimately involved with the implementation and management of the hybrid 
emporium instructional model (see Appendix Table A.1). After analyzing the survey 
responses, in November 2016 we contacted a purposive subgroup of institutions to 
maximize variation in timing and degree of implementation of the model among our 
sample for site visits. 


From January to March of 2017, we conducted site visits at four two-year 
institutions and two four-year institutions (see Table 1). Sites were selected in order to 
achieve maximum variation to explore the potential heterogeneity of student experiences 
in the hybrid emporium model under different conditions. The four community colleges 
were different sizes and in different geographic settings, and their developmental math 
courses were all taught using the hybrid emporium model. The two four-year colleges 
also were different sizes and located in different geographic settings, They also used the 
hybrid emporium model, although not in a traditional prerequisite developmental math 
course format. At the four-year colleges in Tennessee developmental math is only offered 
as a corequisite course. Students in need of developmental math enroll in college-level 
math with an additional section dedicated to a review of the basic concepts. This one- 
credit section is taught using the hybrid emporium model. Through our visits to these six 
institutions, we aimed to capture the diversity of student perspectives at campuses that 
adopted the hybrid emporium model in their developmental math courses. 


At each institution, we conducted hour-long interviews with one administrator 
and one instructor, observed a developmental math class taught using the hybrid 
emporium model, and facilitated a 60-minute focus group with two to eight students. All 
of the students who participated in the focus groups were currently enrolled in a 
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developmental math course taught using the hybrid emporium model. Representatives 
from the math department at each college recruited student participants for the focus 
groups. Focus groups were scheduled in math classrooms during periods when math 
department representatives believed that most students could potentially attend based on 
institutional course-taking patterns. Students were offered an incentive of a $20 Amazon 
gift card for their participation. The open-ended, semi-structured student focus group 
protocol focused on the current experience of students in the classroom, as well as 
perceived benefits and challenges of different aspects of instruction and assessment, with 
particular attention paid to the use of technology (see Appendix B). We structured 
questions to minimize leading students toward particular responses. We asked 
overarching questions and relevant follow-up prompts when necessary. 


Although we took care in the development of the protocol and participant 
recruitment, there was inevitably some selection bias due to the decisions that we made 
regarding our line of questioning and recruitment. Although we endeavored to offer equal 
opportunity for participation for all developmental math students enrolled in a hybrid 
emporium math course, the timing of the focus groups may not have been feasible for all 
students. Participants are largely self-selected, making it more difficult to generalize 
findings from the sample to the population of interest. 


We chose to facilitate focus groups in order to generate a group discussion around 
each question, drawing from the experiences of multiple individuals at the institution 
rather than focusing in-depth on the experiences of one or two students. However, there 
are limitations to using focus groups relative to in-depth interviews. Participants may be 
disinclined to express unpopular opinions or potentially embarrassing experiences. It may 
also be difficult to ensure that the perspectives of underserved or hard-to-recruit 
individuals are represented. However, focus groups better suited our goals for the study 
for a number of reasons. Focus group participants could potentially be more candid, as 
they may be less inclined to provide responses that please the interviewer, which may be 
more likely to occur in a one-on-one setting. Additionally, participants can build on one 
another’s ideas in such a way that constructs a detailed picture of the experience under 
study. And, rather than later finding that interviewees’ experiences differed, the 
interviewer can probe consensus or lack of consensus on topics of interest in real time. 


The primary data leveraged for this paper come from the student focus groups. 
While we must be careful not to overgeneralize findings based on the focus group 
conversations, it is also true that the participants, while small in number, represent a cross 
section of students in a hybrid emporium developmental math course and a range of 
student experiences. The students in our sample (see Table 1) are diverse in terms of their 
gender, race/ethnicity, and age. The focus group participants also vary by initiative and 
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skill level. For example, when asked for the amount of time they spent on math in a given 
week, across the focus groups, students reported spending between 5 and 22 hours per 
week on math outside of class, with a median study time of 9 hours. 
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Table 1 


Description of Site Visit Sample 


Student Focus Group Characteristics 


Undergraduate Total 

Sector Urbanicity Population Persons Race/Ethnicity Gender Age Professional Participants 

Public, two-year Suburb 10,000-19,999 2 Black/A frican- Female (2) Traditional (2) Faculty, department chair, 
American (2) developmental math 

coordinator 

Public, two-year Rural area 5,000-9,999 3 Black/A frican- Female (1), Traditional (2), Faculty, curriculum chair, 
American (2), Male (2) Nontraditional (1) math lab coordinator 
Hispanic (1) 

Public, two-year Small city 1,000-4,999 4 Black/A frican- Female (3), Traditional (4) Faculty, former and current 
American (1), Male (1) department chairs 
Hispanic (1), 
White (2) 

Public, two-year Midsize city 5,000-9,999 6 White (6) Female (3), Traditional (4), Faculty, math lab coordinator 

Male (3) Nontraditional (2) 

Public, four-year Remote town 10,000-19,999 5 Black/A frican- Female (3), Traditional (4), Faculty, department chair, 

or above American (2), Male (2) Nontraditional (1) math lab coordinator 
White (3) 

Public, four-year Large city 5,000-9,999 7 Black/A frican- Female (4), Traditional (6), Faculty, department chair, 

or above American (6), Male (3) Nontraditional (1) math lab coordinator 
White (1) 
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5. Method of Analysis 


For focus groups and interviews, we took notes and audio recorded during the 
sessions. After each site visit, we gathered field notes and discussed the visit. Each focus 
group or interview was transcribed verbatim. We then listened to the audio recordings 
and added field notes in order to integrate the observational data with the audio-recorded 
data. We entered data collection without preconceived notions about student experiences 
and underlying mechanisms and worked to minimize expectations throughout our 
analysis. 


We leveraged a grounded theory analytic approach in order to better understand 
how students felt that their experience of enrolling in developmental math under the 
hybrid emporium model affected their learning and outlook, particularly in comparison 
to their experience of traditional instruction in other courses (Charmaz, 2006; Strauss & 
Corbin, 1990). We used an emergent, open coding approach in order to ground categories 
and themes in the data. We individually coded transcripts and then met to ensure 
systematic application of the coding process across transcripts and to compare and refine 
initial codes. We discussed and clarified the codes, resolved discrepancies, and used 
comparative analysis to identify themes, natural variation, and patterns that emerged 
across the transcripts during iterative rounds of coding. After we identified codes and 
themes in the data, we gathered additional understanding by triangulating data from our 
classroom observations and teacher and administrator interviews. 


After the initial round of coding and discussion, we wrote preliminary memos that 
considered tentative themes that had emerged. In two subsequent rounds of coding, we 
employed axial coding to account for connections across themes. Through our 
discussions, we engaged in theoretical sampling, returning to the data to refine and 
validate our major themes. Once themes were finalized, we engaged in collaborative 
discussion and memo writing to explore how our themes were related to the existing 
empirical and theoretical literature. We integrated our memos and diagrammed 
connections between the emergent themes, identifying domains through an individual and 
team process. We converged on a set of domains that were validated by this iterative, 
triangulated process. Table 2 provides examples of the codes, themes, and domains that 
emerged from our data analysis. For instance, the domain of cognitive accessibility 
includes the emergent theme of increasing access to material and support, in which 
students expressed that the hybrid emporium model provided better opportunities than 
the traditional lecture format to engage with instructional material and concepts. This was 
distinct from themes that emerged in the domain of social accessibility, such as deepening 
relationships between students and instructors, where students described having more 
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substantive interactions and deeper relationships with hybrid emporium instructors than 
instructors in lecture-based courses. We concluded our analyses once we had achieved 
informational saturation, as no new information and themes were emerging from 
additional cases. 
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Table 2 


Examples of Data Coding and Theme Formation 


Example of Raw Interview Data Emergent Theme (Code) Explanation of Code Domain 
“They will tell you what you did wrong and Increasing access to Students find greater opportunities to engage with Cognitive 
where did you mess up wrong, and[youcan material and support instructional material and to find help in accessibility 


choose] ‘Help Me Solve This,’ and ‘Have 
You Solve This,’ ‘Ask Instructor,‘ ‘See an 
Example.”” 


“You get to solve questions and if you need 
more practice you get to repeat it until you 
get it, and you’re certified to know if you’re 
really good at it before you take the test.” 


“You get instant gratification from it. As 
soon as you get done with the test, you don’t 
have to wait [for feedback].” 


“So he’s always there [in class and the math 
lab] to help out and answer any questions 
you’ve got, and he’s always open for emails, 
too.” 


“T can see how passionate and how interested 
she is... . She’s sweet. I like her. She seems 
like a grandma to me, like but give me mom 
speeches.” 


Affording abundant 
opportunities for practice 


Providing immediate 
feedback 


Multiple avenues for 
relationship between 
students and Instructors 


Deepening relationships 
between students and 
instructors 


understanding concepts. 


Students find model structure and platform contents 
provide plentiful problems to practice their skills. 


Students immediately informed of incorrect answers 
and errors are explained. 


Students have more opportunities to interact with 
faculty. 


Students and faculty have substantive interactions 
and form deeper relationships. 


Social accessibility 


“Tt like challenges you to work by yourself 
instead of asking the [instructor or tutor]. 
You can figure out how to do it yourself... . 
You don’t rely on a person.” 


Empowering student 
agency through 
autonomy 


Students have ownership, responsibility for learning 
and resources to meet that responsibility. 


Breaking down 
barriers to math 
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6. Findings 


While focus group participants were enthusiastic about the hybrid emporium 
instructional model overall, they did express some frustrations. Participants’ enthusiasm 
for technology-driven math instruction was moderated by the inconveniences and 
inflexibility of the instructional platform and the pace of the course. In two focus groups, 
students discussed inconvenient aspects of the instructional model, focusing their 
attention on the hours of operation and seating capacity of the math lab, delayed nature 
of faculty responses to questions via email, and occasional malfunctions of the platform, 
particularly during assessments. Despite these criticisms, students in our focus groups 
still expressed overwhelming preference for technology-driven instruction, largely 
related to the ways a hybrid emporium developmental math course breaks down barriers 
to math by increasing cognitive and social accessibility. In this section, we describe 
students’ perceptions of the impact of technology-driven developmental math instruction 
on their academic and psychosocial experiences in college courses. 


The nature of the accessibility afforded by the technology-driven instruction can 
be categorized as cognitive, related to the acquisition of knowledge and understanding, 
or social, related to relationships and emotions. The themes that emerge from students’ 
discussion under the domain of cognitive accessibility are: (1) access to math 
instructional content, (2) abundant opportunities for practice, and (3) immediate feedback 
on their performance. Students also remarked on the social accessibility of the 
mathematics curriculum, citing their (1) satisfaction with multiple avenues for 
relationships with their instructors and (2) deeper relationships with instructors. 


The perceptions of accessibility are particularly important for developmental 
students because of their limited prior success with mathematics content. We note that 
the primary motivation for introducing technology-driven instruction is_ typically 
cognitive—this instructional innovation is intended to improve student understanding and 
mastery of mathematics concepts. The social accessibility of mathematics under the 
colleges’ redesign using the hybrid emporium model constitutes a notable unintended 
consequence. 


Finally, discussion we heard in the focus groups suggests that changes in access 
to content, practice, and support may contribute to overall shifts in students’ perceptions 
of self, math, and even college itself. The focus group data reveal a possible connection 
between cognitive and social accessibility, on the one hand, and students’ perceptions of 
barriers to math and their own agency in engaging in math learning, on the other. 
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Cognitive Accessibility 


Students’ access to additional materials and expert support, abundant 
opportunities for practice, and immediate feedback are the underlying experiences that 
contribute to cognitive accessibility of math under the hybrid emporium model. Our focus 
group participants explained that these features of the model contribute to their ability to 
utilize and feel empowered by the curriculum. 


Increased Access to Material 


Participants consistently remarked that they have more and better access to notes, 
resources, and academic instruction in a hybrid emporium course than in previous math 
courses they have taken. This increased access is reflected in one student’s remark on 
access to material in and outside of the classroom: 


Everything done in class is online for us to look and go back and 
review for when we do our homework. So . . . if we have one 
problem we’re stuck on, we can go back and do that, so that’s 
something I noticed. The technology is a pretty big plus. 


There was a pervasive sentiment among students that they can access and leverage 
material for their learning at any time. Students explained that the web-based platform 
allows for flexible access to content and support materials. Rather than waiting for the 
next class period for clarification or the delivery of new content, the web-based, self- 
paced nature of the course allows students to practice skills and access new material at 
will. Constant accessibility of course content and assignments may be better suited to 
students’ schedules, particularly at two-year institutions where students are more likely 
to work and live off campus. One student said that this accessibility allows him to work 
forward when he has more time and ease off when his schedule is busier. He said, 
“Everything’s already laid out for you, so you sort of know what’s happening. If you got 
to go out of town you can go ahead and do some homework . . . so you don’t fall behind.” 


Under the hybrid emporium model, students also enjoy flexibility in terms of where 
they can complete their assignments and assessments. In addition to completing modules 
and quizzes during the class period, students can continue their math instruction and 
assignment completion wherever they have an internet connection and a compatible device. 
Some students expressed a preference for completing their work during class and in the 
math computer lab, while others said they preferred using personal devices to complete 
their work at home or in other off-campus spaces. One student described dividing his math 
time between home and school: “I go home and do it, too. . . . | have more time at home 
than here, and I get a lot more stuff done. [So] when I come here I get to take the test or 
whatever, and I pass those, and I’m out.” The technology-based instructional platform gives 
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students the freedom to make choices about when, where, and how to learn content, a 
dramatic departure from a traditional lecture-based course. Students frequently referenced 
the math lab as a resource where they can seek help from professors and tutors. 


However, instructional and practice resources are not uniformly accessible from 
home for students at some of the colleges we visited. At one campus—where instructors 
and administrators highlighted that students come from more limited means—students 
mentioned that they are limited in their ability to access these resources depending on 
their technology at home. One student explained that device compatibility is a limitation 
on the flexibility and accessibility of material: “You can do [homework] at home too, but 
it’s really hard, because your computer has to have a certain thing for you to get it. So. . 
. if you’ve got an old computer it’s kind of hard for you to get [logged] in.” For students 
without Internet or a compatible device, progress and practice are limited to campus. This 
can be challenging, particularly for working students for whom the flexibility of 
completing assignments at home may be particularly helpful. 


Students also perceived the pairing of technology-driven instruction and one-on- 
one support as presenting the opportunity to see math done in different, more accessible 
ways. In addition to teacher-led guidance, students also have ready access to technology- 
based resources, math lab tutors, and peers, all of whom may approach or explain math 
concepts and solutions in distinct ways. A number of participants remarked that they are 
able to better connect math to other academic work and their lives outside of school after 
having the opportunity to see math explained in different ways. One student expressed 
appreciation for the alternate perspective her teacher offered in addition to online and 
print resources, saying, “I liked the way she taught. . . . It’s a way that she feels that maybe 
come to us better than what a book has. .. . So she breaks it down, and then online [there’s] 
just more help.” In addition to suggesting that the comprehensive supports of an invested 
teacher paired with online resources contributes to better teaching, this student suggested 
that the change in the instructional norms of the classroom liberates the teacher from 
adhering strictly to a textbook. Another student remarked, “I actually took this class [in 
traditional lecture format]. I dropped it the first time I took it . . . because my professor 
kind of did everything. .. . It wasn’t really teaching us, it was just him rewriting what was 
in the book, and I didn’t learn with him.” Students expressed that the interactive nature 
of the platform, the alternative forms of practice problems, and instructor support all 
afford greater access to the material than in traditional mathematics courses. 


Abundant Opportunities for Practice 


Students overwhelmingly expressed satisfaction with the abundant opportunities 
for practice and the receipt of immediate feedback on practice problems. Instead of 
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working through a finite set of sample problems drawn from worksheets or a textbook, 
the web-based platform can generate nearly infinite problems to hone a particular skill. 
One student commented on the benefits of such adaptive technology: 


In high school they gave you your homework in class and you’d 
finish it, and then you wouldn’t have anything else to do. I mean, 
yeah, that’s okay, but I’d rather go more in-depth with it than I did 
in high school. When we do math homework [on the web-based 
platform] and we got these options to like “Get another question” 
or “Do it again,” it’s like we’re learning from our mistakes. 


If students err while they work through practice problems, they are corrected with 
tailored prompts. When asked whether and how the platform promotes learning, students 
frequently cited the instantaneous feedback. They said that immediate feedback enables 
them to adapt their study methods and pushes them to redouble their efforts. Students 
reported using the instructional resources available to them to varying degrees, but most 
found them to be useful and empowering. 


Students consistently expressed the value of practice-based learning, citing that 
repetition contributes to greater understanding. The practice sections that students are 
required to complete before beginning homework problem sets provide the opportunity 
to complete a seemingly infinite number of practice problems in order to feel more 
confident in their mastery before they are assessed. One student explained that, when 
tackling difficult material, “You have the option of either actually being walked through 
that particular problem, at which point at the end it’Il give you a different problem, or you 
can choose to see an example of a similar problem and then you can even go to the text.” 
For this student, constant access to step-by-step instructions contributes to a better 
learning environment. Students frequently mentioned this process of trial and error as a 
low-risk way of learning from their mistakes. In contrast to lecture-based courses where 
quizzes and exams serve as a single shot for demonstrating knowledge, the practice 
quizzes in the hybrid emporium model allow students to make mistakes and learn from 
those mistakes without academic consequences. One student summarized the benefits of 
continuing to learn from his mistakes: 


Actually going through the exercises and . . . being able to work 
something until you get the right answer just because like once you 
get to that point, it’s kind of— it just clicks. . . . Like there might be 
something that I struggle with, and I might be stuck on a problem 
like number two for maybe like 30 minutes, and then . . . from doing 
it over and over again, I get it. | can breeze through like the next 15 
questions, no problem. 
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Students perceived the wealth of practice problems from the hybrid emporium 
model as one of the primary ways in which the course concepts became more accessible 
to them, both inside and outside of class. 


Immediate Feedback 


Students reported the helpfulness of receiving feedback on their work 
immediately, as the online platforms identify whether a solution is correct and, if not, at 
what stage the solution went awry. This feedback enhances the accessibility of the 
material and contributes to students’ ultimate mastery of math concepts. One student 
remarked: 


And it’s so good to have like immediate feedback of what you did 
instead of doing something in class . .. and you might have to wait 
until the end of class to see if you’ve done something right. . . . It’s 
a lot easier [when you] get immediate feedback of if you did 
something wrong or right, and why. 


When asked as to whether and how the platform promotes their best, deepest 
learning or rote learning through repetition, students frequently cited the instantaneous 
receipt of feedback. One student remarked that the platform facilitates students’ ability to 
identify and thoroughly remediate skills. She reflected, “I think . . . having that immediate 
feedback, like ‘Hey, you got this answer wrong,’ it’s easier to go back and look and see 
where your mistake was and find holes in your understanding of the material.” Students 
expressed confidence that the immediate and comprehensive help they get from the 
platform and instructors in combination is helping them master important skills. 


Students felt that immediate feedback contributes not only to their concept 
mastery, but also to their development and utilization of study skills. The immediate 
feedback they receive on assignments and tests, in particular, gives them the help they 
need to adapt their study methods for success in this and other courses. One student 
pointed out the utility of immediate feedback in the hybrid emporium course for adjusting 
his plan for the semester, as compared to the slower feedback he received in another 
course: 


As soon as you get done with the test, you don’t have to wait... . 
You get to see exactly what you made, and you can plan forward 
from that. So if you bombed it, you can say, “Okay, well, I’m going 
to have to work even harder for the next one starting now.” .. . . [In 
another class] I’m a week and a half from a test [but have not yet 
received the results], so I don’t know if I did fantastic and I should 
keep my study methods or if 1 bombed that one and I need to change 
things up. It’s a lot easier to be able to see if you’re on the right path 
or if you need to step up. 
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The course schedule in traditional lecture-based courses typically does not allow 
for feedback as immediately as in a hybrid emporium course. Overwhelmingly, students 
reported that they like knowing where and why they are going wrong. While at times 
frustrating, this constant feedback loop provides a clear picture for students of where they 
stand in their math course at all times throughout the semester. 


Social Accessibility 


Students also expressed a preference for technology-driven instruction due to 
social connections with classroom actors that contribute to feelings or beliefs that the 
course and material are within their reach. Improvement along this psychosocial 
dimension was largely unintended and unanticipated by those in charge of implementing 
technology-driven instruction at the six colleges. Student discussion of the technology- 
driven instructional experience often focused on their communication, work, and 
relationships with faculty. Faculty-student engagement, both formal and informal, has 
long been identified as a key component of academic integration, which is a primary 
driver of student persistence (Astin, 1993; Chickering & Gamson, 1987; Hurtado & 
Carter, 1997; Pascarella & Terenzini, 2005; Stage & Hossler, 2000; Tinto, 2000; Tinto, 
Goodsell Love, & Russo 1993). 


Our focus groups and observational data suggest that the hybrid emporium model 
disrupts traditional interaction patterns between faculty and students by providing support 
from faculty as “tutor resources” rather than stand-and-deliver sages. The social 
accessibility generated through these relationships may be particularly important for the 
success of academically at-risk students. For the lowest skilled students, previous 
academic experiences, particularly interactions with faculty, may not have been positive. 
The technology-driven instructional model may break down barriers through one-on-one 
or side-by-side interaction. 


Multiple Avenues for Relationships With Instructors 


Under the hybrid emporium model, students find multiple avenues for building 
relationships with faculty. They reported contact in class, in the math lab, through the 
instructional platform, via email, and via mobile application. Students reported that their 
instructors are facile with technology for interaction and that these interactions afforded 
opportunities to connect interpersonally and to share important content-related 
knowledge. 


The instructional platform allows students to contact their teacher at any time 
during instruction or practice. Writing to the instructor directly from the web-based 
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program when a student is having difficulty in solving a problem sends both the student’s 
description of their misunderstanding and a screenshot of the problem they were solving 
and their work completed to that point. Students and teachers both expressed that this was 
a boon for efficiently communicating difficulties and sorting them out without in-person 
contact. When asked whether they ever hold these questions for office hours instead of 
sending them immediately, the majority of students expressed that they do not attend 
office hours, preferring to see teachers in the math lab (where faculty have required 
tutoring time) or communicate virtually. 


Across institutions, students remarked that, through these various forms of 
interaction, they get to know faculty better and feel more comfortable asking them 
questions than in other college classes and in the past. Students consistently expressed 
that technology-based access to faculty enables them to seek help when and how they 
need it. Without the increased technology-based access to faculty, students may not 
otherwise be able to receive personalized assistance with instruction or practice outside 
of the classroom. Fortunately, students found that personalized attention is available to 
them not only in the targeted, immediate feedback they receive from the instructional 
platform but also in timely responses to digital inquiries they send to their instructors. In 
one focus group, two students discussed how they communicate with their shared 
instructor and how accessible she is via email: 


Student 1: If I ever have any questions, if I’m doing it at home, I 
just email her, and she’ll try and get back to me as soon as she 
possibly can. 


Student 2: She’s quick in emailing back too. 


Sentiments like those expressed by these students were common. Students 
communicated that the quick and dedicated responses of faculty are helpful both in their 
understanding of the material and in developing their relationships. Consistent, 
compassionate responses to requests for help confirms faculty investment in student 
development. 


Importantly, this contact between faculty and students is not only in response to 
student questions. Often, students remarked that faculty communication is proactive in 
nature. Some students noted that their technology-driven math instructors are in touch 
with them more frequently outside of the classroom than their other professors. One said, 
“He sends us more emails than . . . all my teachers combined. I at least get like two a day 
from him just about, like, ‘If you want to come [to the lab],’ ‘if you need any help,’ or ‘if 
you need anything.’” Students had positive impressions about the ways in which their 
hybrid emporium instructors are proactive in their outreach and responsive to student 
contact. It’s possible that this high level of accessibility is a product of the additional 
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capacity that faculty have in the absence of lesson design and grading. Another potential 
reason is that faculty who are familiar with an online instructional platform are more 
inclined to reach out via email given their greater developed familiarity with the 
electronic resources. 


In addition to their satisfaction with faculty responses to digital messages and 
proactive contact, students were also impressed with faculty commitment to giving in- 
person feedback. At most institutions, faculty are required to tutor in the math lab for a 
certain number of hours per week, which gives students increased access to instructors 
without the potential barrier of making an appointment or the potential intimidation of 
meeting a faculty member in their office. Overall, students were surprised by the 
accessibility of their professors, particularly given what they had been told about college 
while they were in high school. One student noted her surprise, “I think I like college way 
more than I like high school. . . . [Teachers in high school said that] ‘Teachers are going 
to be mean to you [in college],’ and, ‘They are not really going to care.’ Like, I literally 
came into college thinking that my professors were not going to help me at all.” Students 
consistently expressed that the accessibility of faculty in class, in the math lab, and via 
technology contributes to their instructors being overall more accessible and supportive 
than their high school teachers, and relatively more accessible than other college 
instructors. 


Deeper Relationships With Instructors 


The multiple avenues for contact and the frequency of contact contributes to 
students’ perceptions of deepening interactions between themselves and their instructors. 
Students across institutions spoke of their relationships with their instructors as being an 
instrumental piece of their experience in developmental mathematics. The importance of 
academic support and mentoring relationships with faculty is supported by a long- 
standing literature examining faculty-student interaction (see, for example, Pascarella & 
Terenzini, 2005). 


Regarding their hybrid emporium courses, students described warm, influential 
ties to faculty and felt that faculty are invested in their development and are interested in 
them as individuals. Students across campuses used familial language to describe their 
connections to faculty. One student described how her math instructor’s motherly care 
and attention keeps her on track for academic success, stating, “She’s kind of like amom 
sometimes. She really is! I swear. I get mom looks just from her when I miss school.” 
This student, and others in our focus groups, developed a strong emotional connection 
with her developmental math instructor that she did not feel with faculty in other classes. 
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Students said that the smaller class size in the math lab works to their benefit 
because of the individualized attention they receive from faculty. One student commented 
that, “The smaller the class means the more personal time you get one-on-one with the 
instructor. The bigger the class, the less time they’re going to be able to really sit down 
with you and help you through a problem.” This student identified the utility of the hybrid 
emporium model’s small class size for increasing personalized attention. The math lab 
format itself also gives faculty the opportunity to provide the one-on-one, sit-down 
instruction that students identified as important for their skill development. 


Students appreciated faculty assistance and relationship building so much that a 
number of them recommended that developmental math students be assigned to both their 
developmental and college-level courses with the same faculty member. One student 
commented: 


I have the same professor for both, for [developmental math] and 
for my [college-level] class, which really helps, because you can 
ask her questions about either one when you’re in either class. So if 
you’ ve been having problems with the homework in [college-level], 
you can ask her while you’re in [developmental math], and then 
she’ll do it on the board and go over that lesson again with you in 
[developmental math]. 


Another student from the same institution, upon hearing this, voiced a 
recommendation: “I think it would be beneficial to have the same professor for [both 
classes]. I know that might be hard to do, but . . . then you have the opportunity to ask in 
either direction.” Both students expressed a preference for having the same instructor in 
both their developmental and college-level math classes, pinpointing reasons of rapport 
and shared understanding for this recommendation. 


Faculty-student relationship benefits are not limited to the student’s assigned 
instructor. At institutions where math faculty work in the math lab alongside tutors, 
students also are able to meet and get to know other instructors. Students viewed this as 
an added benefit. Other faculty members might present the information in a way that 
makes more sense to them, or offer helpful perspectives on future coursework or the 
institution on the whole. In this way, students may gain exposure to a faculty member or 
discipline that they would not have learned about otherwise. 


The social accessibility afforded by the model shifts the nature of student-faculty 
relationships, as discussed above. In turn, connections to faculty contribute to changes in 
students’ perceptions of themselves and mathematics, as well as their college and career 
goals. We heard from many students in our sample, perhaps unsurprisingly given their 
enrollment in developmental math, who had previously been intimidated by math or 
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unengaged by the content. However, students across campuses noted that they connected 
with faculty during early challenging moments in the course and were able to harness 
faculty support and the nature of the instructional model to turn around their performance. 


Students remarked that their professors have shaped their relationship with the 
course material. Instructors play a role in lowering barriers to math and shaping students’ 
academic pathways. One older student reported, “Before I got here, math was not 
something I liked doing. I hated math. I literally cried through the first two weeks of 
school... . And it was when I met with my math support teacher during her office hours 
and took the time to get a little bit of one-on-one that everything clicked. And now, I 
mean, I want to get a math degree.” Increasing the cognitive and social accessibility of 
mathematics can have powerful consequences for perceptions of self and of the material, 
and faculty relationships are central to this evolution. 


Breaking Down Barriers to Math 


Students felt that technology-driven instruction lowers barriers that they had faced 
in prior math instruction, which leads to students feeling more empowered and more like 
success is within their control. One way this manifests is in students changing their math- 
related behavior. Students cited the requirements of technology-driven math courses as 
an important impetus that pushes them outside of their typical patterns of behavior. 
Students from multiple institutions suggested that as a function of being required to spend 
time in the math lab, they spend more time on math work and leverage important 
resources. They reported that they ultimately began to see math as less intimidating. The 
requirement and their subsequent exposure leads to opportunity and habituation such that 
their perceptions of their abilities and environment change. One student commented: 


I enjoy the fact that we were kind of forced into the math lab, 
because otherwise I would have been way too intimidated to go into 
it. Kind of like a girl going to the gym to lift weights: Like we want 
to do it, but we’re afraid we’re going to look stupid. . . . Because I 
hear math lab, and I think, “There’s a bunch of geeky people in 
there, and that I’m ... going to stick out like a sore thumb because 
I don’t know what pi is,” you know? And so being in there for my 
math support class kind of helped to make that a less intimidating 
environment. 


This student said that previously she had not felt welcome in math-centered spaces 
because of her low relative skill level and identity as a non-math person. The requirement 
to go to math lab and the welcoming environment and support she received there have 
flipped the story: She now feels empowered to use the math lab, a space that she would 
have perceived as “off limits” otherwise. 
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Empowering Student Agency Through Autonomy 


Students were eager to share about how the format of the hybrid emporium model 
puts the onus on students to get through the material and seek additional help when 
needed, which may serve as a mechanism by which students perceive barriers to math to 
be lower. Students felt that the self-paced nature of the technology-driven modules is 
more conducive to the different rates at which individuals learn material than the 
traditional classroom environment. One student conveyed that he felt more at ease with 
the pace and better-equipped to tackle the material as a result of the autonomy he was 
granted to learn the material: 


You don’t have to rush as much. You make sure you really get it 
before you move on... . If you don’t get it, you won’t progress, 
and—I mean, like it’s not somebody breathing down your neck 
trying to get you to do this, do this, do this. Make sure you get it, 
understand it, move on. Because we all learn at different levels. It’s 
not just one class pace. 


Another student supported this sentiment when reflecting on challenges he faced 
keeping up with material in a more traditional math course: “I don’t really understand in 
the classroom what we’re doing, because it’s just so fast. I can’t really keep up. I’m not 
able to process everything that’s really going on, and it’s like once you miss a step when 
you’re taking notes for math, you don’t understand it.” Several students reported feeling 
less overwhelmed by the pace in a hybrid emporium classroom when compared to the 
pace of some of their more traditional courses. 


While students remarked that the nature of hybrid emporium instruction gives them 
ownership over their learning, this self-paced and self-motivated approach can be a double- 
edged sword. One community college student discussed the benefits and challenges of 
students being given the responsibility of pacing their movement through the material: 


It’s like a blessing and a curse. ... This [course] is specific lessons 
that you have to do, which you could get them all done a week in 
advance, or you could fall behind on one lesson and it makes the 
other lessons harder on you. 


Increased autonomy along with increased access to instruction, practice material, 
and support appear to shape hybrid emporium students’ perceptions of self and of math, 
affecting their actions as a student, both in their developmental math courses and other 
college courses. 
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7. Discussion and Conclusion 


The students in our sample reported cognitive and social accessibility benefits of 
technology-driven instruction. And they perceived having a greater sense of 
responsibility for their own learning as well. Our analysis reveals that these benefits are 
rooted in access to material, support, and relationships that empower students to succeed 
and offer the feeling that success is within their locus of control. Students reported that 
the hybrid emporium model increases their access to instructional material and support, 
provides abundant opportunities for practice, and connects them with immediate 
feedback. These underlying experiences contribute to the cognitive accessibility of math 
in the redesigned curriculum. Our focus group participants explained that these features 
contribute to their ability to utilize the platform and feel empowered under the hybrid 
emporium model of instruction. These experiences align with the goal of the model: to 
improve the student learning experience in developmental math courses. 


While the purpose of this instructional reform is focused on cognitive 
accessibility, we also observed an unintended consequence—the model serves to open up 
multiple avenues to connect with faculty and to deepen faculty-student relationships. Our 
analysis suggest that the model disrupts traditional interaction patterns between faculty 
and students by reducing formal barriers between them. This evidence, drawn from the 
student perspective, stands in contrast with previous findings indicating that teachers 
perceive that technology-driven models may contribute to declines in classroom 
interaction and support (Natow, Reddy, & Grant, 2017; Groff & Mouza, 2008). 


Our findings suggest that the hybrid emporium model takes the primary onus of 
instruction off the faculty, giving them flexibility to better support students inside and 
outside the classroom. Faculty spend the class period floating between students to provide 
one-on-one support. The hybrid emporium model provides instruction and abundant 
opportunities for practice, so faculty are able to concentrate their efforts on “spot 
treatment” of challenges and misunderstandings that arise as students work independently 
through the modules. The reduced instruction and grading load may allow them to be 
more responsive via email, instructional platform messaging, and mobile application. At 
the majority of institutions, students noted their appreciation that faculty are required to 
staff the math lab, as this also gives them access to faculty tutoring support. Interacting 
with faculty in these more informal settings allows students to feel more at ease seeking 
out help. 


The social accessibility of math as it relates to improved interactions with faculty 
is supported by the faculty-student engagement literature. Faculty-student engagement is 
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a key component of academic integration and contributes to student persistence (Astin, 
1993; Chickering & Gamson, 1987; Hurtado & Carter, 1997; Pascarella & Terenzini, 
2005; Stage & Hossler, 2000; Tinto, 2000; Tinto, Goodsell Love, & Russo, 1993). 
Students who arrive at college underprepared for postsecondary mathematics work are 
most at risk for dropout and disengagement from academic material. Student-faculty 
engagement may reinforce students’ place in the academic fabric of the institution. 
Instructors can be change agents in the classroom: Their instruction and engagement with 
students can change the cognitive relationship that students have with academic content 
and can shape students’ beliefs, emotions, and perceptions about math (Wang, Sun, & 
Wickersham, 2017). Given the dropout risk profile of students in developmental courses, 
the social accessibility generated through these relationships may be particularly 
important for student success. 


Students in developmental coursework represent the plurality of community 
college enrollees and are the most vulnerable for dropout. Oftentimes, under alternative 
course structures, students find themselves caught in a prescribed developmental math 
sequence, unable to proceed to college-level coursework. The instructional redesign 
involved in transitioning to the hybrid emporium model, which reshapes perceptions of 
the cognitive and social accessibility of content and supports, should help to bolster 
students and empower them to succeed. Yet recent research finds negative effects of 
technology-driven instruction on students’ subsequent academic performance (Boatman, 
2019; Kozakowski, 2019). Therefore, we question why our participants articulated such 
benefits and general enthusiasm when quasi-experimental estimates from Tennessee 
suggest that, in comparison to students taking traditional developmental math courses, 
hybrid emporium students have lower pass rates in their subsequent college math courses 
and fewer credits completed over time (Boatman, 2019). 


We find some guidance on this matter in the literature. It is possible that students’ 
positive experiences with the hybrid emporium instructional model feed into a 
mechanism that is unrelated to academic performance. College student experiences of 
classroom autonomy may be more closely related to motivational factors than to 
performance (Garcia & Pintrich, 1996). Although students may not perform better in the 
course or remain enrolled in college longer, the experience of autonomy may foster 
intrinsic motivation and self-efficacy, which may prove useful in other collegiate contexts 
not captured by traditional measures of student performance such as course grades or 
persistence. 


Additionally, the perceived benefits regarding cognitive and social accessibility 
may not yield large enough changes in self-efficacy, math content knowledge, or other 
benefits to boost performance for students in developmental math. Meta-analytic analysis 
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of 39 studies of self-efficacy and academic performance found that self-efficacy beliefs 
account for approximately 14 percent of the variance in students’ academic performance 
and approximately 12 percent of the variance in academic persistence (Multon, Brown, 
& Lent, 1991). However, much of the literature focuses on self-efficacy in elementary 
and secondary school students or among postsecondary students in college-level 
coursework. Additional investigation into the explicit relationship between self-efficacy 
and course grades among young adult and adult students in developmental coursework 
would shine light on a potentially important mechanism for understanding performance 
in this student population. Further, simply because students report that math concepts are 
more accessible to them in a hybrid emporium course does not necessarily mean that 
students are improving in their understanding of these concepts. While students may feel 
as if math is easier to learn in a technology-driven classroom, a comprehensive 
assessment at the beginning and end of the course would provide evidence on actual as 
opposed to perceived learning gains. 


The limitations of this study are similar to other case study analyses. First, the 
institutions where we conducted site visits and the students with whom we spoke 
constitute a purposive sample. While we selected institutions from a range of geographies 
and asked administrators to recruit a diverse group of students, there are limits to the 
generalizability of our findings. Selection also plays a role with respect to who teaches 
hybrid emporium model math courses. At some institutions, teaching developmental 
courses is required, while at others faculty select their course assignments. 
Notwithstanding these limitations, we find consistent, emergent themes based on how 
students in our focus groups reported their experiences in the classroom. 


In this study we did not focus on the experience of students through the lenses of 
gender, race/ethnicity, or other characteristics due to the small sample of students. We 
acknowledge that students’ backgrounds play an important role in their in-class 
experiences and thus are likely central to the experience of any instructional innovation. 
Future research should focus in particular on the role of race/ethnicity, gender, age, 
socioeconomic status, and other student characteristics on student experiences in and 
perceptions of technology-driven developmental math instruction (Wang et al., 2017). 


While this paper focuses on student experiences rather than student outcomes, the 
outcomes-focused literature is mixed on the effects of this and similar models that use an 
online instructional model (Alpert, Couch, & Harmon, 2016; Bowen, Chingos, Lack, & 
Nygren, 2014; Xu & Jaggars, 2013). While some studies have null or positive effects, on 
balance, the evidence suggests that technology-driven remediation is negatively 
associated with college mathematics success and future attainment. The results of the 
outcomes-focused literature are not consistent with the perceptions that students have of 
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their experiences and raise interesting questions about the lack of alignment between the 
positive student experience of hybrid emporium math and the lackluster outcomes 
(Boatman, 2019; Kozakowski, 2019). It may be the case that there are other, larger 
barriers to developmental math learning that are not addressed by this model. The 
accessibility boons of this instructional model may engage and empower students while 
still falling short of what developmental mathematics coursework must accomplish in 
terms of skills and content mastery. Our findings suggest that there are positive 
unintended consequences of technology-driven mathematics instruction, but these 
changes do not necessarily lead to improved course performance. 


Students’ positive perceptions of the cognitive and social accessibility of their 
hybrid emporium coursework also raise the question: Are negative medium- and long- 
term relationships between such models and student outcomes due to drop-offs in the 
quality of instruction, supports, and relationships in subsequent math classes? Given the 
positive experiences of students with increased access to material and feedback and 
increased time and more avenues to connect with faculty and receive individualized 
support, we urge faculty and policymakers to consider ways in which curricular redesigns 
may be employed to put more material in the hands of students and preserve instructor 
time for greater personalized support and stronger relationship development. The 
continued study of students’ experiences in developmental math coursework, particularly 
in classrooms and at institutions that employ innovative instruction, is critical to the 
success of postsecondary students and of interest to the institutions that serve them. 
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Appendix A: Supplementary Table 


Table A.1 


Description of 2016 Survey Sample and Participants 


Math 
Remediation 
Rate (%) For 
Undergraduate First-Time 
Sector Urbanicity Population Students* Survey Completer 
Public, two-year Remote town 1,000—-4,999 61-70 Developmental math 
coordinator 
Public, two-year Remote town 5,000-9,999 51-60 Director of institutional 
effectiveness and 
planning 
Public, two -year Remote town 5,000-9,999 51-60 Developmental math 
coordinator 
Public, two -year Suburb 10,000-19,999 41-50 Department chair and 
developmental math 
coordinator 
Public, two -year Suburb 5,000—9,999 61-70 Developmental math 
coordinator 
Public, two -year Rural area 5,000-9,999 61-70 Curriculum chair and 
math lab coordinator 
Public, two -year Rural area 5,000-9,999 41-50 Department chair 
Public, two -year Small city 1,000-4,999 61-70 Dean, math & science 
Public, two -year Small city 1,000-4,999 61-70 Department chair 
Public, two -year Small city 5,000-9,999 Developmental math 
coordinator 
Public, two -year Midsize city 5,000-9,999 51-60 Department chair 
Public, two -year Large city 5,000-9,999 71-80 Dean, division of math 
and natural sciences 
Public, two -year Large city 5,000-9,999 71-80 Department chair 
Public, four-year or above Remote city 10,000-19,999 41-50 Faculty member and 
math lab coordinator 
Public, four-year or above Small city 10,000—19,999 Faculty member and 
math lab coordinator 
Public, four-year or above Midsize city 10,000—19,999 Developmental math 
coordinator 
Public, four-year or above Midsize city 20,000 and above Faculty member 
Public, four-year or above Large city 20,000 and above Faculty member 
Public, four-year or above Large city 5,000-9,999 71-80 Faculty member and 


math lab coordinator 


*Survey completers were asked, “Approximately what percentage of your institution’s first-time students are assigned 
to developmental math” with choices 0-10, 11-20, 21-30, etc. At the four-year colleges in 2016, developmental math 
was offered only as part of a corequisite math course (college-level combined with developmental). Four of the survey 
respondents did not distinguish developmental math from college-level math, and did not complete this question. 
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Appendix B: Semi-Structured Protocol for Student 
Focus Groups 


Please describe the math class in which you are currently enrolled (instructors, 
classroom setting, required hours, assignments, number of students per section, etc.) 
a. Describe what a typical session in lab/class looks like. What do you do? How 
many people are there? 
b. Do you ask questions? Are your instructor’s answers helpful? 
c. When you raise your hand, how long is the wait for your instructor to get to 
your question? 
How do you use technology in class? 
a. How does this differ from your other classes this term? From previous math 
courses? 
What is most beneficial to you about the format of this class? What is the most 
challenging? 
How many lab/classroom hours are required? 
a. Should all students be required to spend the same amount of time in the lab? 
If not, why not? If so, how many hours should be required? 
b. Does your teacher encourage you to come to lab hours? If so, how? 
How have you used the math skills that you have learned from this course in your 
other classes or in your life outside of school? 
Does the material in this class repeat the material you have learned in other courses 
in high school or in college? If so, is this repetition helpful or boring? 
How does this classroom model promote your best learning as an individual student? 
Does the individual pacing of the class make you work through the material more or 
less quickly than in other math classes you’ve taken? 
Do you have more or less interaction with your instructor than you have had in 
previous math courses or other courses at this institution? 


. How does the difficulty of the work and time required for this course compare to a 


traditional three-credit course? 
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